The LION Way: Machine Learning plus Intelligent Optimization by Roberto Battiti & Mauro Brunato

Author:Roberto Battiti & Mauro Brunato [Battiti, Roberto]
Language: eng
Format: epub
Publisher: Roberto Battiti and Mauro Brunato
Published: 2014-02-25T22:00:00+00:00

Gist

Recurrent neural networks with feedback loops allow the transition from "mathematical functions" (feedforward networks) to full-fledged dynamical systems with evolution in time and internal memory.

Machine learning for recurrent neural networks is very hard, in particular for derivative-based methods. The many cycles involved can cause derivatives to explode or to vanish.

The recently proposed reservoir computing (RC) and extreme learning machines (ELM) methods take a radical approach, in a way contrary to that of deep learning, by creating vast amounts of random building blocks (random features), and limiting learning to a final linear combination layer. A specific problem "taps" the useful building blocks in the reservoir and weighs them appropriately to get the final solution.

Given the difficulties of realizing deep derivative-based techniques with noisy biological neural hardware, the success of this brute-force "randomized construction plus final tuning" approach gives new hope to explain parts of our brain and to engineer fast and flexible learning machines.

We are happy to live in a sparkling research period, when crazy and wildly different ideas advance the frontier of ML and neural networks through spectacular plot twists and paradigm changes.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.